On the complexity analysis of randomized block-coordinate descent methods

On the Complexity Analysis of Randomized Block-Coordinate Descent Methods

In this paper we analyze the randomized block-coordinate descent (RBCD) methods proposed in [11, 15] for minimizing the sum of a smooth convex function and a blockseparable convex function, and derive improved bounds on their convergence rates. In particular, we extend Nesterov’s technique developed in [11] for analyzing the RBCD method for minimizing a smooth convex function over a block-separ...

Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function

In this paper we develop a randomized block-coordinate descent method for minimizing the sum of a smooth and a simple nonsmooth block-separable convex function and prove that it obtains an ε-accurate solution with probability at least 1− ρ in at most O((n/ε) log(1/ρ)) iterations, where n is the number of blocks. This extends recent results of Nesterov [Efficiency of coordinate descent methods o...

Randomized Sparse Block Kaczmarz as Randomized Dual Block-Coordinate Descent

We show that the Sparse Kaczmarz method is a particular instance of the coordinate gradient method applied to an unconstrained dual problem corresponding to a regularized `1-minimization problem subject to linear constraints. Based on this observation and recent theoretical work concerning the convergence analysis and corresponding convergence rates for the randomized block coordinate gradient ...

On the Convergence of Block Coordinate Descent Type Methods

In this paper we study smooth convex programming problems where the decision variables vector is split into several blocks of variables. We analyze the block coordinate gradient projection method in which each iteration consists of performing a gradient projection step with respect to a certain block taken in a cyclic order. Global sublinear rate of convergence of this method is established and...

Block coordinate descent methods for semidefinite programming